Exemplary embodiments of the invention relate to a device that detects an object on a vehicle seat, comprising a camera and a processing unit that generates a two-dimensional representation of the object.
There are various techniques and systems for detecting, identifying or classifying the occupants of a motorized vehicle. Furthermore, there are various techniques for the recognition of the head position of vehicle occupants. Information from such techniques mainly serves as data for the adaptation of occupant restraint systems to the type, size, or seat position of the occupants.
According to the Federal Motor Vehicle Safety Standard (FMVSS) 208 set out by U.S National Highway Transportation and Safety Administration (NHSTA), starting with the 2006 model year nearly 100 percent of all automobiles sold in the US must have the ability to automatically control the deploying power of airbags based on crash severity, occupant type, and size, as well as on seat belt usage. As manufacturers began to develop various occupant detection systems, vision techniques have attracted much attention due to their superior adaptability to various vehicle cabin environments as compared to other mechatronic methods.
In recent years, a number of optical approaches have been studied to resolve the airbag suppression decision problem. These studies can be classified into two categories depending on the number of cameras used in the system. In the earlier versions of occupant detection systems, single camera approaches were in demand due to the high cost of imaging sensors. However, such monocular systems did not provide sufficient 3D information necessary for functions such as the out-of-position detection, which is a supplementary task guaranteeing low risk deployment according to the position/pose of the passenger. As a consequence, the majority of occupant detection systems became more dependent on stereo vision techniques that use two cameras. Faced with the increasing demand for various vision-based in-vehicle applications, the growing number of cameras employed has come under serious scrutiny.
In the following, some of the most important drawbacks of existing optical approaches to occupant detection are enumerated.                Stereo camera systems use more than one camera, and multiple cameras significantly increase the hardware cost.        Many approaches use supplementary light sources for illuminating the vehicle interior, which increases the hardware cost as well.        Due to the theoretical limitation of stereo vision, two cameras must be adjacent to each other, which makes it difficult to design a compact system module.        Stereo vision systems are extremely sensitive to any physical impacts distorting the geometry of the involved cameras.        Since the 3D reconstruction of most stereo vision techniques is essentially based on a texture correspondence matching, most stereo vision approaches fail to reconstruct the shape of a target object when the object does not have textures on its surface.        To obtain a satisfactory depth resolution, most stereo vision techniques employ sub-pixel spatial resolution which increases the overall system cost by requiring wider data bandwidth as well as faster computing power.        Intensive searching and triangulation processes, which are essential for stereo vision, influence the system cost.        Most stereo vision techniques require complicated camera calibration.        
Exemplary embodiments of the present invention provide a single camera system able to generate 3D information of a vehicle seat occupant, which can be achieved using minimal supplementary active illumination.
Exemplary embodiments of the present invention provide a device that detects an object on a vehicle seat, comprising a camera and a processing unit that generates a two-dimensional representation of the object. The device also comprises at least two illumination sources, which are positioned in such a manner that the object can be illuminated from different directions, and the processing unit can generate a three-dimensional representation of the object from two two-dimensional representations, each generated under different illumination conditions.
Typically, a device according to the present invention comprises a single camera by which an image sequence is captured. Different illumination conditions can be caused by dimming, switching, or movement (displacement and/or rotation) of the illumination sources. For simplicity, according to an exemplary embodiment of the present invention, different illumination conditions are caused by switching the illumination sources on and off, i.e., different illumination conditions are caused by different combinations of active illumination sources, particularly by a different number of active illumination sources.
In accordance with one exemplary embodiment of the present invention the processing unit can generate a three-dimensional representation of the object from two two-dimensional representations, each generated under different illumination conditions, whereby the different illumination conditions are caused by a different number of active illumination sources. The generation of a three-dimensional representation from two two-dimensional representations can include a generation from more than two two-dimensional representations.
An occupant detection system according to the present invention can classify an object in a vehicle for facilitating the airbag control module. Classification can be carried out based on object features which are extracted from the original images and/or from the three-dimensional object representation and/or from intermediate results. The classification can be used to assign the object to a target occupant class.
According to an exemplary embodiment of the present invention, the number of target occupant classes in such a classification is limited. The target occupant classes are reduced to three classes being a front facing child seat, a rear facing child seat and an adult.
Therefore, an exemplary embodiment of the present invention includes a processing unit that can assign the detected object to one out of at least two object classes, where the at least two object classes are selected from a plurality of object classes comprising the three following object classes:                a child-seat arranged in forward driving direction,        a child-seat arranged in backward driving direction,        an adult.        
The result of the object classification can be used to simplify other tasks. For this purpose, the result can be signaled to another task.
According to an exemplary embodiment of the present invention, the processing unit signals the object class to which the device on the vehicle seat has been assigned to a task, which is executed either by the processing unit itself or by another device, whereby the fact and/or manner of the execution of this task depends on the object class to which the device on the vehicle seat has been assigned. Particularly, according to an exemplary embodiment of the present invention an out-of-position detection is only activated, if the object is classified as an adult.
In a method according to the present invention, different illumination conditions within an image sequence are generated by at least two light sources. Ambient illumination fluctuations and/or shadows are eliminated.
According to an exemplary embodiment of the present invention ambient illumination fluctuations in the image sequence are eliminated by a so-called Double flash technique.
A shadowless image can be composed from the image sequence by simulating a virtual light source having an infinite extent. This is suitably achieved by a so-called ShadowFlash technique.
The boundary information of an object in the resulting shadowless image can be extracted and the 3D surface of the object can be reconstructed.
Finally, according to an exemplary embodiment of the present invention, a multi-dimensional feature vector is defined and is utilized for classification purposes. The feature vector can use 2D and/or 3D information.
By the present invention a novel framework for a vehicle occupant detection system is provided. The performance of a device according to the present invention is comparable to stereo vision systems. Thus, a device according to the present invention can serve as an alternative solution to binocular vision systems offering some substantial advantages.                A device according to the present invention is economical compared to stereo vision techniques, since only a single camera is needed. The market price of the required switchable illumination sources is usually lower than that of a camera.        Compared to stereo vision approaches less computing power is necessary for a device and/or method according to the present invention. The complexity of the algorithms used in the proposed system is normally lower than the complexity of any stereo vision algorithm.        Since the proposed system is not based on texture correspondence matching, the performance of the 3D reconstruction is completely independent of the textural property of a target object.        The degree of freedom for locating the camera and light sources is much higher than with most stereo vision techniques.        In a method according to the present invention the depth resolution is only a function of the spatial resolution of the camera. Therefore, no sub-pixel imaging is necessary.        The complexity, in particular the hardware complexity, of a system according to the present invention is normally lower than the complexity of a system based on stereo vision.        
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of the invention when considered in conjunction with the accompanying drawings.